This goal of this document is to offer an easy way to process the outputs of the praise reward system and perform an analysis of the resulting token reward distribution. It should be considered a work-in-progress.
Since praise gets valued on a scale, we can take a look at how often each value of the scale gets assigned by quantifiers. Note: how to process the duplicate scores? For now, just delete them.
We can now take a look at the distribution of the received praise rewards. You can toggle the inclusion of the different sources by clicking on the legend.
We can also take a look at the distribution of the people giving praise.
Now for something more fun: let's surface the top "praise flows" from the data. Thanks to @inventandchill for this awesome visualization! On one side we have the top 20 praise givers separately (modifiable by changing the variable n_senders), on the other the top 25 receivers (modifiable by changing the variable n_receivers). The people outside the selection get aggregated into the "REST FROM" and "REST TO" categories.
Let's take a closer look at the quantification process and and see if we can spot any problems:
To aid the revision process, we highlight disagreements between quantifiers.
Here we generate a table which sorts the praise by the size of the spread between the highest and lowest scores. It gives us an overview of the spread distribution.
For an exhaustive list, take a look at the exported file "praise_outliers.csv" .
This is a visual aid. ATTENTION! If there are several praise instances with similar spread and quant score, all but one end up "hidden" on the chart.
Let's see the range of praise scores every quantifier gave to see the behavior difference of quantifiers.
To interpret the box plot:
Bottom horizontal line of box plot is minimum value
First horizontal line of rectangle shape of box plot is First quartile or 25%
Second horizontal line of rectangle shape of box plot is Second quartile or 50% or median.
Third horizontal line of rectangle shape of box plot is third quartile or 75%
Top horizontal line of rectangle shape of box plot is maximum value.
Praise instances with disagreements in duplication are collected in 'results/duplication_examination.csv'. To compare, look at the last 4 columns: 'DUPLICATE MSG 1/2/3' and 'ORIGINAL MSG'.
Praise instances with disagreements in dismissal are collected in'results/dismissal_disaggreed.csv'. You can further look into who dismissed and who did not.